Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing
نویسندگان
چکیده
We present studies of the attention and time, or engagement, invested by crowd workers on tasks. Consideration of worker engagement is especially important in volunteer settings such as online citizen science. Using data from Galaxy Zoo, a prominent citizen science project, we design and construct statistical models that provide predictions about the forthcoming engagement of volunteers. We characterize the accuracy of predictions with respect to different sets of features that describe user behavior and study the sensitivity of predictions to variations in the amount of data and retraining. We design our model for guiding system actions in real-time settings, and discuss the prospect for harnessing predictive models of engagement to enhance user attention and effort on volunteer tasks.
منابع مشابه
Likely to stop? Predicting Stopout in Massive Open Online Courses
Understanding why students stopout will help in understanding how students learn in MOOCs. In this report, part of a 3 unit compendium, we describe how we build accurate predictive models of MOOC student stopout. We document a scalable, stopout prediction methodology, end to end, from raw source data to model analysis. We attempted to predict stopout for the Fall 2012 offering of 6.002x. This i...
متن کاملTowards Feature Engineering at Scale for Data from Massive Open Online Courses
We examine the process of engineering features for developing models that improve our understanding of learners’ online behavior in MOOCs. Because feature engineering relies so heavily on human insight, we argue that extra effort should be made to engage the crowd for feature proposals and even their operationalization. We show two approaches where we have started to engage the crowd. We also s...
متن کاملWhat's Hot in Crowdsourcing and Human Computation
The focus of HCOMP 2014 was the crowd worker. While crowdsourcing is motivated by the promise of leveraging people’s intelligence and diverse skillsets in computational processes, the human aspects of this workforce are all too often overlooked. Instead, workers are frequently viewed as interchangeable components that can be statistically managed to eek out reasonable outputs. We are quickly mo...
متن کاملMSRC at TREC 2011 Crowdsourcing Track
Crowdsourcing useful data, such as reliable relevance labels for pairs of topics and documents, requires a multidisciplinary approach that spans aspects such as user interface and interaction design, incentives, crowd engagement and management, and spam detection and filtering. Research has shown that the design of a crowdsourcing task can significantly impact the quality of the obtained data, ...
متن کاملSolver engagement in knowledge sharing in crowdsourcing communities: Exploring the link to creativity
There is increasing interest in online communities as a channel of innovation for companies. However, research to date has been limited on how to create a compelling virtual experience to inspire contestants to make novel and creative contributions. This examination is critical as the online space grows more crowded. This paper draws from job engagement theory and the theory of job design to de...
متن کامل